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Background 


This  is  a  brief  description  of  a  long-term  research 
program  in  optimization  theory  and  practice  conducted 
within  the  Operations  Research  Department  of  the 
Naval  Postgraduate  School  (NPS).  Our  program  is 
unique  in  that  our  theoretical  research  is  predominantly 
motivated  by  pressing  real-world  problems  for  which 
we  are  allowed  very  little  time  to  organize  and  produce 
viable  solutions.  The  principal  sources  of  these  prob¬ 
lems  are  our  research  sponsors  and  graduate  students 
with  inflexible  due  dates.  We  show  how  beneficial  the 
interaction  of  research  and  instruction  at  NPS  can  be 
by  use  of  a  typical  thesis  project.  Over  the  years,  our 
optimization  research  program  has  embraced  many 
fields  of  application  and  theoretical  development  out  of 
immediate  necessity  to  support  our  demanding  clients. 
We  have  also  served  as  an  important  conduit  between 
our  civilian  and  military  collaborators  at  other  institu¬ 
tions  who  have  assisted  us  in  our  research.  Projects  in 
the  program  have  been  supported  by  the  Office  of 
Naval  Research,  the  National  Science  Foundation, 
Department  of  Energy,  Defense  Logistics  Agency  and 
Joint  Chiefs  of  Staff. 


The  Operations  Research  (OR)  Program  at  NPS  is 
unique.  Operations  Research,  defined  by  Webster  as 
the  application  of  scientific  and  especially  mathematical 
methods  to  the  study  and  analysis  of  complex  overall 
problems,  was  born  of  military  necessity  during  World 
War  II.  Thus,  it  is  not  surprising  that  the  OR  program 
at  NPS  is  the  oldest  and  one  of  the  largest  such 
graduate  engineering  programs  in  the  U.S.  Students  are 
usually  commissioned  officers  from  uniformed  services 
of  the  U.S.,  and  from  Australia,  Canada,  Egypt,  Ger¬ 
many,  Greece,  Israel,  Korea,  Mexico,  Turkey  and 
many  other  allied  countries.  Typical  students  have  an 
undergraduate  engineering  degree,  five  to  ten  years  of 
successful  professional  experience,  and  the  aptitude  and 
diligence  to  make  them  the  envy  of  any  university. 
Most  of  our  students  pursue  the  Master’s  Degree  in  a 
program  lasting  eight  to  nine  quarters.  Their  first  year 
is  occupied  by  core  coursework.  The  second  year 
begins  with  a  half-quarter  experience  tour,  during 
which  each  student  is  transferred  to  an  operations 
analysis  activity  somewhere  (e.g.,  Washington,  D.C., 
Hawaii,  Iceland,  etc.)  and  given  actual  work  to  do  in 
his  or  her  primary  professional  specialty.  Returning 
from  this  taste  of  reality,  advanced  coursework,  elec¬ 
tives,  and  a  thesis  complete  the  program.  Although  a 
handful  of  doctoral  students  contribute  to  our  research 
programs,  it  is  principally  the  Master’s  thesis  require¬ 
ment  which  occupies  us. 

Theses  frequently  focus  on  real  problems  which 
the  student  will  face  immediately  following  graduation. 
A  thesis  topic  may  be  theoretical,  or  quite  practical,  but 
it  must  be  accepted  by  an  advisor  and  a  second  reader; 
these  faculty  members  often  act  as  collaborators  with 
the  thesis  student,  providing  an  enriched  environment 
for  all. 
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According  to  Webster,  optimize  is  making  as  per¬ 
fect,  effective,  or  functional  as  possible.  Optimization  in 
Operations  Research  evokes  a  scientific  and  especially 
a  mathematical  connotation  for  this  definition.  Compu¬ 
tation  provides  the  experimental  evidence  with  which 
mathematical  theory  is  scientifically  tested.  Optimiza¬ 
tion  is  one  of  the  fundamental  areas  of  study  in  Oper¬ 
ations  Research  at  NPS. 

A  Case  Study 

To  illustrate  the  singular  nature  and  pace  of 
our  research  program  in  optimization,  consider  a  typi¬ 
cal  thesis  project.  Navy  LT  Clarke  Goodman  complet¬ 
ed  his  core  studies  and  spent  his  experience  tour  in 
Norfolk,  VA.  While  working  on  his  assigned  project, 
LT  Goodman  became  fascinated  with  an  unrelated 
activity  occupying  a  signficant  number  of  people  and 
directly  affecting  the  entire  U.S.  Atlantic  Fleet.  LT 
Goodman  witnessed  the  preparation  of  an  Annual  Em¬ 
ployment  Schedule.  The  employment  schedule  directs 
the  activities  of  all  air,  marine,  submarine,  and  surface 
untis  for  the  following  quarter,  and  informs  such  units 
of  planned  activities  for  a  year  in  advance.  Having  been 
the  object  of  such  employment  schedules  for  years,  LT 
Goodman  was  naturally  interested  in  the  reasoning 
underlying  their  formulation  and  in  possibly  improving 
their  effectiveness  and  fairness.  His  questions  were 
unofficially  entertained  by  members  of  the  planning 
staff. 

At  NPS,  LT  Goodman  convinced  Professor  Ke¬ 
vin  Wood  that  the  employment  schedule  problem  was 
a  worthy  topic,  and  enlisted  my  support  as  a  second 
reader.  LT  Goodman  and  Wood  worked  together  to 
formalize  the  reasoning  for  and  render  a  tractable 
model  of  the  employment  scheduling  process.  They 
decided  to  restrict  their  attention  to  major  surface 
combatants  (e.g.,  frigates,  destroyers,  cruisers,  carriers, 
etc.)  and  to  use  an  actual  unclassified  annual  schedule 
as  a  mechanism  to  test  their  model.  LT  Goodman 
returned  to  Norfolk  to  formally  extract  additional  data 
and  to  receive  criticism  and  additional  guidance  on  his 
proposed  approach. 

LT  Goodman  then  devised: 

1.  A  data  base  for  all  surface  combatants  in  the  U.S. 
Atlantic  Fleet  (over  100  ships),  including  the  weapons 
systems  and  capabilities  of  each  ship  as  well  as  restric¬ 
tions  for  fixed  commitments  (e.g.,  yard  periods,  re¬ 
quired  exercises,  or  even  administrative  fiats). 

2.  A  data  base  for  all  major  events  in  the  following 
year  (e.g.,  fleet  exercises,  deployment,  etc.),  including 
the  complementary  requirements  of  each  activity  (e.g., 
numbers  and  types  of  ships,  weapons  systems,  substitut¬ 
ability  of  units,  mission  effectiveness,  etc.) 


3.  A  schedule  generator  which  efficiently  produces 
large  numbers  of  alternate  employment  schedules  for 
each  ship,  recognizing  the  unique  attributes  of  each 
ship  and  honoring  its  fixed  commitments.  A  crucial 
feature  of  this  generator  is  a  mechanism  to  estimate  the 
contribution  of  each  ship  to  the  mission  at  hand,  given 
its  other  commitments  as  well  as  its  crew  and  weapons 
systems  endowment. 

4.  An  optimization  model  to  select  from  the  large 
number  of  candidate  ship  schedules  a  particular  set  of 
schedules  which  satisfies  event  requirements  with 
maximal  effectiveness. 

5.  A  report  writer  to  produce  an  annual  employment 
schedule  in  its  published  format,  as  well  as  analyses  of 
the  criteria  by  which  such  schedules  are  evaluated 
(e.g.,  at-sea  time,  time  between  deployments,  etc.) 

The  optimization  model  can  be  illustrated  by  the 
simplified  example  in  Figure  1 . 


Figure  1 
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The  object  is  to  select  from  this  matrix  a  set  of  columns 
which  collectively  has  exactly  one  1  in  each  row.  This 
deceptively  simple  model  is  known  as  a  set  partition 
problem.  LT  Goodman  used  each  column  to  represent 
an  alternate  annual  ship  schedule,  and  each  row  to 
represent  mutual  exclusion  among  all  schedules  for 
each  ship  (ship  rows),  or  appropriate  assignment  of 
proper  ships  and  weapons  to  each  event  (require¬ 
ment/event  rows).  Furthermore,  each  of  his  columns 
has  a  value  (not  shown  in  Figure  1)  which  contributes 
additively  to  the  value  of  the  fleet  employment  sched¬ 
ule  and  this  total  value  must  be  maximized. 
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As  the  number  of  rows  and  columns  in  the  matrix 
grows,  the  resulting  set  partition  optimization  problem 
becomes  an  integer  linear  program  of  great  combina¬ 
toric  difficulty.  To  appreciate  the  task  at  hand,  consider 
an  artificial  case  in  which  only  one  hundred  randomly 
generated  columns  need  be  evaluated.  There  are  2  to 
the  power  100  (as  a  decimal  number,  this  is  more  that  1 
followed  by  31  zeroes)  different  candidate  solutions  to 
try.  Because  LT  Goodman  expected  to  generate  set 
partitions  with  tens  of  thousands  of  rows  plus  columns, 
we  contacted  Prof.  Glenn  Graves  at  UCLA  whom  we 
knew  to  be  solving  similar  problems  in  crew  scheduling 
for  United  Airlines.  Graves  agreed  to  help  us  cope  with 
whatever  Goodman  produced. 

Over  the  next  few  weeks,  the  data  structures  and 
logic  of  the  schedule  generator  were  refined  technical¬ 
ly  to  efficiently  produce  columns  restricted  to  those 
most  likely  to  be  used  in  an  optimal  set  partition  (fleet 
employment  schedule).  Considerable  modelling  effort 
was  devoted  to  the  evaluation  of  each  candidate  ship 
schedule  so  that  the  multiple  criteria  of  combat  effec¬ 
tiveness,  crew  fatigue,  substitutability  of  assets,  equita¬ 
ble  distribution  of  workload  among  ships,  etc.,  could  all 
be  effectively  addressed.  In  concert,  we  developed  a 
set-partition  solver  based  on  existing  general-purpose 
experimental  optimization  software  (called  the  X-sys- 
tem  by  Brown  and  Graves)  and  some  special-purpose 
procedures  contributed  by  a  prior  Master’s  thesis  by 
Marine  CAPT  Dan  Bausch.  The  enriched  set  of  (about 
20  thousand)  columns  yielded  by  the  generator  was 
attacked  in  many  experiments  aimed  at  discovering  and 
exploiting  special  structure  in  the  set-partition  matrices 
and  thus  reducing  solution  efforts. 

LT  Goodman  and  the  professors  enlisted  to  sup¬ 
port  him  assembled  in  less  than  two  months  a  prototyp- 
ic  system  which  accepts  a  user-friendly  input  script 
with  ship  and  event  descriptions,  and  manual  pre¬ 
assignments  of  ships  to  events,  and  automatically  pro¬ 
duces  a  complete  annual  employment  schedule  in  less 
than  a  minute  of  time-shared  computer  time.  Objective 
comparison  of  the  quality  of  the  schedules  produced 
automatically  and  those  produced  manually  and  subse¬ 
quently  published  for  actual  use  reveal  significant  im¬ 
provements  in  all  criteria.  The  sheer  speed  of  response 
of  the  system  suggests  that  schedulers  could  employ  it 
as  an  interactive  tool  to  incorporate  non-quantifiable 
criteria  or  frequent  changes  in  actual  schedules.  Tech¬ 
nical  accomplishments  of  this  work  have  been  present¬ 
ed  in  international  research  society  meetings  and  will 
appear  in  a  research  paper  in  the  open  literature. 


Optimization  Research 
Program 

Our  objective  for  the  overall  optimization  research 
program  at  NPS  is  to  qualitatively  improve  the  effi¬ 
ciency  with  which  we  can  solve  real-life  problems 
using  optimization.  Our  methods  include  theoretical  as 
well  as  extensive  empirical  investigation  of  problems 
presented  to  us,  and  of  problems  posed  as  wholly 
artificial  test  cases. 

Many  of  the  classes  of  problems  with  which  we 
must  deal  (e.g.,  integer  programs)  are  believed  to  be 
intractable  in  the  sense  that  there  is  strong  theoretical 
evidence  (but  as  yet  no  proof)  that  there  can  never  be 
an  algorithm  to  efficiently  compute  optimal  solutions 
for  all  problem  instances.  Despite  the  daunting  theoret¬ 
ical  computational  complexity  of  these  problems,  the 
gloomy  prospects  for  success  are  rarely  borne  out. 
Real-life  problems  seem  to  exhibit  special  properties 
which  render  them  solvable  by  an  approach  which 
identifies  and  exploits  the  specializations.  This  has  led 
us  to  a  guiding  Principle  of  Optimization:  careful  model¬ 
ling  of  real-life  systems  that  are  shaped  by  strong  eco¬ 
nomic,  physical,  social,  and/or  rational  forces  yields 
problems  that  are  dramatically  easier  to  solve  than 
others.  Following  this  principle,  we  seek  means  to 
effectively  identify,  extract,  express,  and  exploit  in  our 
models  and  solution  methods  problem  features  reflect¬ 
ing  these  underlying  forces. 

Solution  methods  can  be  characterized  as  direct,  in 
which  the  problem  is  solved  outright,  or  indirect,  in 
which  related  sub-problems  are  solved  and  their  solu¬ 
tions  used  to  assemble  a  solution  to  the  original  prob¬ 
lem. 

Direct  methods  benefit  from  factorization,  in  w'hich 
some  subset  of  the  problem’s  rows  or  columns  are 
identified  as  exhibiting  special  structure  and  treated 
separately  by  the  solution  method.  Networks  are  an 
excellent  example  of  such  structure,  and  we  have  found 
that  networks  are  contained  within  most  real-life  mod¬ 
els. 
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Indirect  methods  break  up  the  problem  into  more 
manageable  pieces,  solve  the  pieces  and  assess  the 
solutions  achieved  to  see  if  they  solve  the  original 
problem.  If  not,  the  pieces  can  be  altered,  and  resolved, 
perhaps  many  times,  to  obtain  solutions  closer  to  that 
required  by  the  original  problem.  Frequently,  solutions 
achieved  by  indirect  means  are  approximate  and  our 
goal  is  to  achieve  minimal  error  at  reasonable  cost.  The 
classical  indirect  approach  is  decomposition,  or  its  cur¬ 
rently  fashionable  heuristic  simplification  called  La- 
grangean  relaxation.  We  have  used  both  approaches, 
especially  when  the  manageable  pieces  thus  produced 
are  networks.  In  addition  to  repeatedly  solving  the 
manageable  pieces,  decomposition  usually  requires  re¬ 
peated  solution  of  a  restricted  master  problem  to  which 
constraints  are  successively  added  to  ensure  conver¬ 
gence.  Lagrangean  relaxation  can  be  arranged  to  omit 
this  difficult  step,  but  our  experience  is  that  conver¬ 
gence  is  not  so  easy  to  prove  theoretically  or  achieve  in 
practice. 

Both  direct  and  indirect  methods  are  improved, 
especially  with  integer  models,  if  we  modify  them  to 
allow  violation  of  each  constraint  at  some  cost.  Thus, 
the  constraints  become  aspirations,  or  goals,  and  the 
choice  of  whether,  or  by  how  much  to  violate  con¬ 
strains  is  made  by  the  model.  We  refer  to  models  with 
linear  violation  penalties  as  elastic  models,  and  in  our 
experience  there  is  much  to  recommend  them.  At  first 
glance,  elastic  models  appear  to  cheat  by  violating 
constraints.  However,  solutions  which  offer  enormous 
improvement  via  some  slight  constraint  violation  can 
be  very  attractive  in  real-world  problems.  Our  research 
has  produced  new  solution  methods  for  network  mod¬ 
els,  such  as: 

□  Assignment  (e.g.,  men  to  jobs,  weapons  to  targets, 
etc.) 

□  Transportation  (e.g.,  rail  shipment  from  depots  to 
units) 

□  Capacitated  Transshipment  (e.g.,  capacitated  phys¬ 
ical  distribution  systems) 

□  Generalized  Networks  (e.g.,  amplification  and/or 
attenuation  in  transit) 

□  Integer  Generalized  Networks  (e.g.,  flows  restrict¬ 
ed  to  one  of  two  values) 

as  well  as  for  other  models,  such  as: 

□  Linear  Programming 

□  Mixed  Integer  Linear  Programming 

□  Nonlinear  Programming 

□  Quadratic  Assignment  (e.g.,  travelling  salesman 
problems) 


In  cases  for  which  our  computer  programs  do  not 
require  extensive  field  support  (which  we  are  not  able 
to  provide),  we  have  made  the  programs  available  to 
other  academic  researchers.  Thus,  problems  formerly 
at  the  threshold  of  the  state-of-the-art  have  been  re¬ 
duced  to  routine  features  within  even  more  ambitious 
models. 

The  optimization  group  at  NPS  has  also  agreed  to 
apply,  when  possible,  our  more  complex  experimental 
computer  systems  to  problems  supplied  to  us  with  a 
publication  release  for  our  research  findings. 

A  partial  topical  list  of  applications  of  optimization 
which  we  have  pursued  includes: 

□  Physical  distribution  system  design 

□  Energy  production,  distribution  and  consumption 

□  Ship  scheduling  and  fleet  employment 

□  Munitions  procurement,  storage  and  shipment 

□  Production  scheduling 

□  Search,  detection  and  surveillance 

□  Vehicle  dispatching  and  routing 

□  Target  assignment 

□  Mobilization 

□  Manpower 

□  Network  reliability 

□  Engineering  design 

□  Real-time  process  optimization 

□  Capital  budgeting 

An  important  role  for  NPS  and  its  research  pro¬ 
grams  is  to  serve  as  a  medium  for  communication 
among  academic,  military,  and  civilian  researchers. 
The  list  above  gives  ample  evidence  of  the  general 
usefulness  of  optimization.  We  have  scrupulously 
shown  how  to  schedule  supertankers  with  the  same 
techniques  used  for  Naval  surface  combatants,  how  to 
bake  crackers  and  cookies  or  produce  ammunition, 
how  to  dispatch  trucks  or  allocate  sorties  to  military 
aircraft,  how  to  build  capital  portfolios  or  plan  flight 
tests  of  new  fighter  aircraft,  and  so  forth  for  many 
application  areas. 

We  are  concerned  that  we  can  often  solve  models 
much  more  efficiently  than  we  can  implement,  validate, 
support  or  understand  them.  Accordingly,  in  concert 
with  our  modelling  research,  we  have  developed  sever¬ 
al  prototypic  user-friendly  model-building  and  solu¬ 
tion-interpretation  systems.  (Necessity  leads  to 
invention:  a  misspelling-tolerant  system  for  naming 
model  entities  was  developed  following  a  comical  late- 
night  terminal  session  in  which  students  futilely  sought 
transportation  data  for  Albuquerque.) 
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Even  more  ambitiously,  the  entire  process  of  for¬ 
mulation,  model  statement,  data  specification,  and 
model  validation  and  interpretation  has  been  formally 
addressed  by  Professor  Gordon  Bradley,  Ph.D.  student 
Army  MAJ  Bob  Clemence,  and  Professor  Art  Geof- 
frion  (UCLA).  Geoffrion  has  specified  a  formal  ap¬ 
proach  called  structured  modelling,  and  Bradley  and 
MAJ  Clemence  have  extended  the  concept  and  imple¬ 
mented  a  system  called  LEXICON  to  illustrate  the 
efficacy  of  the  approach.  They  have  extended  this 
work  by  developing  the  concept  of  typing  and  type 
validation  that  allows  automatic  verification  that  the 
model  correctly  expresses  the  modeler’s  intentions. 

All  these  research  activities  have  contributed  to  a 
rather  large  optimization  research  software  suite  which 
enables  quick  response  to  new  modelling  challenges. 
Yet,  there  are  always  demands  for  solutions  to  larger, 
more  difficult  models.  But  such  is  the  nature  of  re¬ 
search,  each  breakthrough  leads  to  more  challenges  for 
our  program  to  pursue, 
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